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EXAMINER'S AMENDMENT 

1 . An examiner's amendment to the record appears below. Should the changes 
and/or additions be unacceptable to applicant, an amendment may be filed as provided 
by 37 CFR 1 .31 2. To ensure consideration of such an amendment, it MUST be 
submitted no later than the payment of the issue fee. 

Authorization for this examiner's amendment was given in a telephone interview 
with Nema Berezny (Reg. No. 57,706) on April 6, 201 1 . 

The application has been amended as follows: 

Claim 10: Please replace claim 10 with the following claim language: 

10. A computer-implemented method of capturing and treating content using a 
computer system having a processor, memory, and data storage subsystems, the 
computer-implemented method comprising: 

setting a mode of operation to a content capture mode for interpreting user input 
for the purpose of selecting an on-screen region of a display, and receiving a path 
drawn by a user, the path defining boundaries of the selected on-screen region of the 
display, wherein pixels comprising one or more graphical elements representing a first 
set of one or more textual characters are displayed in the selected on-screen region; 

capturing the pixels displayed within the selected on-screen region, and storing 
the captured pixels in an image file; 
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switching the mode of operation to an annotation mode in response to a user 
command; 

receiving an annotation drawn by the user on the display, wherein the received 
annotation is implemented using a plurality of tools via a toolbar, the toolbar appearing 
after the selecting an on-screen region; 

obtaining context information for the one or more graphical elements by 
automatically applying text recognition to the annotation, and storing the results of the 
text recognition as context information via the computer system, wherein certain context 
information comprises preventive measures that limit an association of history with the 
one or more graphical elements based upon digital rights management licenses, and 

obtaining additional context information by extracting the first set of one or more 
textual characters and extracting a second set of textual characters displayed in 
proximity with the first set, 

wherein the context information and the additional context information are 
automatically stored in association with the image file. 

Claim 27: Please replace claim 27 with the following claim language: 

27. A computer-implemented method of context harvesting from selected content 

using a computer system having a processor, memory, and data storage subsystems, 

the method: 

receiving a path drawn on a display via user input, the drawn path defining 
boundaries of a selected on-screen region of the display, the selected on-screen region 
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comprising a plurality of pixels, wherein a displayed content of the selected on-screen 
region includes textual data and underlying data; 

capturing the plurality of pixels and associated underlying text and associated 
links to embedded information of the on-screen region by capturing only complete 
characters or words; 

storing the captured pixels as an image file; 

automatically extracting a character or word from the textual data and extracting 
complete sentences based upon punctuation as context information in response to 
determining that the displayed content of the on-screen region includes the textual data 
via the computer system; 

pointing a first pointer from the context information to the displayed content; 

automatically extracting a property of the underlying data as additional context 
information in response to determining that the displayed content of the on-screen 
region includes the underlying data via the computer system, the property comprising at 
least one of: a file name, a file identifier, a uniform resource locator (URL), a uniform 
resource identifier (URI), a folder name, or meta-data; 

determining a window associated with the selected on-screen region, and 
automatically extracting a uniform resource identifier (URI) from a name property or a 
value property of the window as additional context information; 

pointing a second pointer from the additional context information to the displayed 
content; and 
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storing the extracted context information and additional context information in 
association with the image file via the data storage subsystem, such that the context 
information is accessible when viewing the image file. 

Claim 28: Please replace claim 28 with the following claim language: 

28. A computer-implemented method of context harvesting from selected content 
using a computer system having a processor, memory, and data storage subsystems, 
the method comprising: 

receiving a display of content comprising at least one of textual data, an 
executable object, a file, or a link to remote content; 

setting a mode of operation to a content capture mode; 

receiving a path drawn on the display which defines boundaries of a selected on- 
screen region of the content via user input; 

capturing and storing the selected content as an image file via the data storage 
subsystem; 

switching the mode of operation to an annotation mode; 
receiving an annotation on the display via the user input; 
performing text recognition on the received annotation as context information; 
pointing a first pointer from the selected content to the context information; 
automatically determining via the computer system that the content displayed 
within the on-screen region includes textual data and underlying data; 
automatically extracting as additional context information: 
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a character or word from the textual data of a bounded region determined to be 
included in the on-screen region, wherein the character or word that lies completely 
within the bounded region is extracted as additional context information, and the 
character or word that does not lie completely within the bounded region is not extracted 
as additional context information, and 

a property of the underlying data determined to be included in the on-screen 
region, the property comprising at least one of: a file name, a file identifier, a uniform 
resource locator (URL), a uniform resource identifier (URI), a folder name, or meta-data; 
and 

relating the selected content to the additional context information via a separate 
linking structure comprising identifying information; 

pointing a second pointer from the selected content to the additional context 
information; and 

storing the context information and the additional context information in 
association with the image file via the data storage subsystem, such that the context 
information is accessible when viewing the image file. 

Claim 36: Please replace claim 36 with the following claim language: 
36. The computer-implemented method of claim 10, further comprising: 
navigating through the second sets of textual characters that reside within the path 
drawn by the user, and subsequently navigating through any sets of textual characters 
that reside outside the path drawn by the user. 
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Reasons for Allowance 

2. The following is an examiner's statement of reasons for allowance: 

3. Regarding claim 1 0, the prior art fails to teach the combination of the claimed 
limitations. Zacks et al. (US 2004/0123131) teaches a image metadata processing 
system and method, in which he teaches preventing access to certain metadata 
(context) associated with an image (graphical element) (claims 16 and 17). However, 
Zacks and all other prior art fail to explicitly teach certain context information comprising 
preventive measures that limit an association of history with the one or more graphical 
elements based upon digital rights management licenses, in combination with the 
claimed setting, capturing, switching, receiving, and obtaining steps. 

4. Regarding claim 27, the prior art fails to teach the combination of the claimed 
limitations. Oppermann (US 6,334,157) teaches obtaining a URI property as context 
information (col. 11 lines 1-9, 55-60, col. 12 lines 55-60, col. 13 lines 51-60). However, 
Oppermann and all other prior art fail to explicitly teach determining a window 
associated with the selected on-screen region, and automatically extracting a uniform 
resource identifier (URI) from a name property or a value property of the window as 
additional context information, in combination with the claimed receiving, capturing, 
storing, extracting and pointing steps. 

5. Regarding claim 28, the prior art fails to teach the combination of the claimed 
limitations. Dawe (US 7,042,594) teaches determining if any handwriting lies within a 
region (col. 7 lines 11-16) and Bayiates (US 7,467,134) teaches a option of including 
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only text that is completely within a region or including all text that intersects a region 
(col. 10 lines 53-55). However, Dawe and Bayiates and all other prior art fail to explicitly 
teach automatically extracting as additional context information a character or word from 
the textual data of a bounded region determined to be included in the on-screen region, 
wherein the character or word that lies completely within the bounded region is 
extracted as additional context information, and the character or word that does not lie 
completely within the bounded region is not extracted as additional context information, 
in combination with the claimed receiving, setting, capturing, switching, performing, 
pointing, determining, extracting, relating and storing steps. 

Any comments considered necessary by applicant must be submitted no later 
than the payment of the issue fee and, to avoid processing delays, should preferably 
accompany the issue fee. Such submissions should be clearly labeled "Comments on 
Statement of Reasons for Allowance." 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to ALICIA M. LEWIS whose telephone number is 
(571)272-5599. The examiner can normally be reached on Monday - Friday, 9 - 6:30, 
alternate Friday off. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Charles Rones can be reached on 571-272-4085. The fax phone number 
for the organization where this application or proceeding is assigned is 571-273-8300. 
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Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 



/Alicia M Lewis/ 
Examiner, Art Unit 2164 
April 6, 2011 



